The present invention relates to two dimensional (2D) to three dimensional (3D) medical image registration, and more particularly, to deep learning based 2D/3D medical image registration.
Two dimensional (2D) to three dimensional (3D) image registration is an important technology in medical imaging and image-guided interventions. 2D/3D image registration can be used to bring pre-operative 3D medical image data and intra-operative 2D medical image data into the same coordinate system to facilitate accurate diagnosis and/or provided advanced image guidance. For example, the pre-operative 3D medical image data generally includes computed tomography (CT), cone-beam CT (CBCT), magnetic resonance imaging (MRI), and/or computer aided design (CAD) models of medical devices, while the intra-operative 2D medical image data is typically X-ray images.
2D/3D image registration is typically achieved using intensity-based methods. In such intensity-based 2D/3D image registration methods, in order to register a 3D X-ray attenuation map provided by CT or CBCT (or converted from another imaging modality), a simulated X-ray image, referred to as a digitally reconstructed radiograph (DRR), is derived from the 3D attenuation map by simulating the attenuation of virtual X-rays. An optimizer is then employed to maximize an intensity-based similarity measure between the DRR and X-ray images. Intensity-based methods are able to achieve high registration accuracy, but suffer drawbacks including long computation time and small capture range. Because intensity-based methods involve a large number of evaluations of the similarity measure, each requiring heavy computation in rendering the DRR, such methods typically result in running times greater than one second, and therefore are not suitable for real-time applications. In addition, because the similarity measures to be optimized in intensity-based methods are often highly non-convex, the optimizer has a high chance of getting trapped into a local maxima, which leads to such methods having a small capture range in which high registration accuracy can be achieved.